{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Quant Backtesting Workflow in Marquee\n",
    "\n",
    "This notebook demonstrates how to implement continuous portfolio optimizations using GS Quant."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 1: Authenticate and Initialize your Session \n",
    "\n",
    "First you will import the necessary modules and add your client id and client secret."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import datetime as dt\n",
    "import time\n",
    "from math import copysign\n",
    "\n",
    "import pandas as pd\n",
    "\n",
    "from gs_quant.datetime.relative_date import RelativeDate\n",
    "from gs_quant.markets.position_set import Position, PositionSet\n",
    "from gs_quant.markets.portfolio import Portfolio\n",
    "from gs_quant.markets.portfolio_manager import PortfolioManager\n",
    "from gs_quant.markets.securities import Asset, AssetIdentifier\n",
    "from gs_quant.markets.report import FactorRiskReport, ReturnFormat\n",
    "from gs_quant.models.risk_model import FactorRiskModel\n",
    "from gs_quant.session import GsSession\n",
    "from gs_quant.target.common import PositionSetWeightingStrategy\n",
    "\n",
    "from gs_quant.target.hedge import CorporateActionsTypes\n",
    "from gs_quant.markets.optimizer import (\n",
    "    OptimizerStrategy,\n",
    "    OptimizerUniverse,\n",
    "    AssetConstraint,\n",
    "    FactorConstraint,\n",
    "    SectorConstraint,\n",
    "    OptimizerSettings,\n",
    "    OptimizerConstraints,\n",
    "    OptimizerObjective,\n",
    "    OptimizerType,\n",
    ")\n",
    "\n",
    "pd.set_option('display.width', 1000)\n",
    "\n",
    "client = None\n",
    "secret = None\n",
    "\n",
    "## External users must fill in their client ID and secret below and comment out the line below\n",
    "# client = 'ENTER CLIENT ID'\n",
    "# secret = 'ENTER CLIENT SECRET'\n",
    "\n",
    "GsSession.use(client_id=client, client_secret=secret)\n",
    "\n",
    "print('GS Session initialized.')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Quick Hint:\n",
    "If you don't already have a portfolio set up, follow the instructions in  [01_Create_Backcasted_Portfolio](../Portfolios/01_Create_Backcasted_Portfolio.ipynb) to create a snapshot portfolio and backcast it. \n",
    "Alternatively, if you have a historical position set, you can also create a historical portfolio in [02_Create_New_Historical_Portfolio](../Portfolios/02_Create_New_Historical_Portfolio.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 2: Load your Portfolio"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "Load your portfolio and define hedge parameters\n",
    "\n",
    "##### Note - The start date for your continuous optimization must be **after** the first position set uploaded in your source portfolio."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "port_id = 'YOUR PORTFOLIO ID'  # put in your existing portfolio id\n",
    "start_date = dt.date(2024, 1, 2)\n",
    "hedge_notional_pct = 1.0\n",
    "universe = ['SPX']\n",
    "rebalance_freq = '1m'\n",
    "risk_model_id = 'AXIOMA_AXWW4M'\n",
    "apply_factor_constraints_on_total = True"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "port_object = Portfolio.get(port_id)\n",
    "port_manager = PortfolioManager(port_id)\n",
    "print(f'Using portfolio {port_object.name} with id {port_id} as a source portfolio')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 3: Create a Portfolio for your Optimization\n",
    "\n",
    "\n",
    "You will first create a portfolio with a custom name and share it with a list of emails.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "opt_port_object = Portfolio(name=f'{port_object.name} - L/S Hedge')\n",
    "opt_port_object.save()\n",
    "opt_port_id = opt_port_object.id\n",
    "print(f'Created portfolio {opt_port_object.name} with id {opt_port_id}')\n",
    "opt_port_manager = PortfolioManager(opt_port_id)\n",
    "opt_port_manager.share(['your.email@yourcompany.com', 'your.colleague@yourcompany.com'], admin=True)\n",
    "opt_port_manager.set_tag_name_hierarchy(['source'])\n",
    "\n",
    "opt_risk_report = FactorRiskReport(risk_model_id=risk_model_id)\n",
    "opt_risk_report.set_position_source(opt_port_id)\n",
    "opt_risk_report.save()\n",
    "\n",
    "opt_port_manager.update_portfolio_tree()\n",
    "print(f'Using portfolio {opt_port_object.name} with id {opt_port_id} as the optimization portfolio')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Risk and performance information about your source portfolio is required in order to run the continuous optimization. \n",
    "\n",
    "We'll first make sure that your source portfolio's factor risk and performance reports have finished successfully."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print('Ensuring performance and risk calculations are complete on the source...')\n",
    "perf_report = port_manager.get_performance_report()\n",
    "perf_report.get_most_recent_job().wait_for_completion()\n",
    "\n",
    "risk_report = port_manager.get_factor_risk_report(risk_model_id=risk_model_id)\n",
    "risk_report.get_most_recent_job().wait_for_completion()\n",
    "\n",
    "factor_exposure_data = risk_report.get_factor_exposure(start_date=start_date, end_date=start_date)\n",
    "factor_exposure_map = factor_exposure_data.to_dict(orient='records')[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 4: Setting Up Optimization Parameters\n",
    "\n",
    "Now that you have a position set, you can get a hedge according to your liking.\n",
    "`prepare_factor_constraints` below will pull constraints defined at a portfolio level to apply on the hedge."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "def prepare_factor_constraints(factor_constraints, port_factor_exposure_map):\n",
    "    \"\"\"Given factor constraints defined on the total portfolio and factor exposures of the core portfolio,\n",
    "    return constraints to be applied on the hedge\"\"\"\n",
    "    new_constraints = []\n",
    "    for fc in factor_constraints:\n",
    "        old = fc.max_exposure\n",
    "        new = port_factor_exposure_map.get(fc.factor.name, 0) - fc.max_exposure\n",
    "        print('Changing factor constraint for ', fc.factor.name, 'from ', old, 'to ', new)\n",
    "        new_constraints.append(\n",
    "            FactorConstraint(fc.factor, port_factor_exposure_map.get(fc.factor.name, 0) - fc.max_exposure)\n",
    "        )\n",
    "    return new_constraints"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that you have a position set, you can get a hedge according to your liking. We have put in some sample settings below. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "hedge_universe = OptimizerUniverse(\n",
    "    assets=[Asset.get(a, AssetIdentifier.BLOOMBERG_ID) for a in universe],\n",
    "    explode_composites=True,\n",
    "    exclude_corporate_actions_types=[CorporateActionsTypes.Mergers],\n",
    ")\n",
    "\n",
    "risk_model = FactorRiskModel.get(risk_model_id)\n",
    "\n",
    "asset_constraints = [\n",
    "    AssetConstraint(Asset.get('MSFT UW', AssetIdentifier.BLOOMBERG_ID), 0, 5),\n",
    "    AssetConstraint(Asset.get('AAPL UW', AssetIdentifier.BLOOMBERG_ID), 0, 5),\n",
    "]\n",
    "\n",
    "# Specify the constraints on factor exposure of the Total Optimized Portfolio\n",
    "factor_constraints = [\n",
    "    FactorConstraint(risk_model.get_factor('Size'), 0),\n",
    "]\n",
    "\n",
    "if apply_factor_constraints_on_total:\n",
    "    hedge_factor_constraints = prepare_factor_constraints(factor_constraints, factor_exposure_map)\n",
    "else:\n",
    "    hedge_factor_constraints = factor_constraints\n",
    "\n",
    "sector_constraints = [SectorConstraint('Energy', 0, 30), SectorConstraint('Health Care', 0, 30)]\n",
    "constraints = OptimizerConstraints(\n",
    "    asset_constraints=asset_constraints,\n",
    "    factor_constraints=hedge_factor_constraints,\n",
    "    sector_constraints=sector_constraints,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 5: Continuous Optimization Loop\n",
    "\n",
    "With our initial setup done and settings configured, we are now ready to launch a flow that will continuously optimize our portfolio at our desired frequency.\n",
    "\n",
    "We will take the positions from the previous rebalance, utilize performance analytics to get the latest positions, and then optimize the portfolio again. We will then update the portfolio with the new positions and schedule reports for the next rebalance date.\n",
    "\n",
    "This operation of moving your portfolio forward using performance analytics relies solely on availability of the underlying assets. \n",
    "\n",
    "##### Note - The Optimizer is designed to work with prices adjusted for events such as dividends, splits, and corporate actions. This means it reflects the true economic value of assets after such adjustments. In contrast, our portfolio analytics operate on the raw (unadjusted) closing prices to provide a more detailed, day‐to‐day picture of historical trade activity. To avoid inconsistencies, we recommend passing position sets based solely on weight (which is independent of price adjustments) to derive the optimization output also in terms of weights"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "port_manager = PortfolioManager(port_id)\n",
    "start = start_date\n",
    "max_end = RelativeDate(\"-1b\", dt.date.today()).apply_rule(exchanges=['NYSE'])\n",
    "start_time = time.time()\n",
    "rebal = start_date\n",
    "\n",
    "optimizer_runs = []\n",
    "\n",
    "while rebal < max_end:\n",
    "    print(f'Moving to rebalance date {rebal}')\n",
    "\n",
    "    source_port_perf_report = port_manager.get_performance_report()\n",
    "    latest_source_port_pos_data = source_port_perf_report.get_portfolio_constituents(\n",
    "        start_date=rebal,\n",
    "        end_date=rebal,\n",
    "        fields=['quantity', 'grossWeight'],\n",
    "        prefer_rebalance_positions=True,\n",
    "        return_format=ReturnFormat.JSON,\n",
    "    )\n",
    "    latest_source_port_exp = source_port_perf_report.get_gross_exposure(start_date=rebal, end_date=rebal)[\n",
    "        'grossExposure'\n",
    "    ][0]\n",
    "    latest_source_position_set = PositionSet(\n",
    "        date=rebal,\n",
    "        reference_notional=latest_source_port_exp,\n",
    "        positions=[\n",
    "            Position(\n",
    "                asset_id=p['assetId'],\n",
    "                identifier=p['assetId'],\n",
    "                # We recommend using gross weight to find your reference weight, like below\n",
    "                weight=copysign(p.get('grossWeight', 0), p.get('quantity', 0)),\n",
    "                tags=[{'source': 'Portfolio'}],\n",
    "            )\n",
    "            for p in latest_source_port_pos_data\n",
    "        ],\n",
    "    )\n",
    "    settings = OptimizerSettings(\n",
    "        notional=hedge_notional_pct * latest_source_position_set.reference_notional,  # 40% of your original portfolio\n",
    "        allow_long_short=True,\n",
    "    )\n",
    "\n",
    "    if factor_constraints and apply_factor_constraints_on_total:\n",
    "        source_port_risk_report = port_manager.get_factor_risk_report(risk_model_id=risk_model_id)\n",
    "\n",
    "        factor_exposure_data = risk_report.get_factor_exposure(\n",
    "            start_date=latest_source_position_set.date, end_date=latest_source_position_set.date\n",
    "        )\n",
    "        factor_exposure_map = factor_exposure_data.to_dict(orient='records')[0]\n",
    "        hedge_factor_constraints = prepare_factor_constraints(factor_constraints, factor_exposure_map)\n",
    "    else:\n",
    "        hedge_factor_constraints = factor_constraints\n",
    "\n",
    "    constraints = OptimizerConstraints(\n",
    "        asset_constraints=asset_constraints,\n",
    "        factor_constraints=hedge_factor_constraints,\n",
    "        sector_constraints=sector_constraints,\n",
    "    )\n",
    "    strategy = OptimizerStrategy(\n",
    "        initial_position_set=latest_source_position_set,\n",
    "        constraints=constraints,\n",
    "        settings=settings,\n",
    "        universe=hedge_universe,\n",
    "        risk_model=risk_model,\n",
    "        objective=OptimizerObjective.MINIMIZE_FACTOR_RISK,\n",
    "    )\n",
    "    print('Optimizing...')\n",
    "    # Using adjusted prices in the Optimizer to reflect post-corporate action values\n",
    "    strategy.run(optimizer_type=OptimizerType.AXIOMA_PORTFOLIO_OPTIMIZER)\n",
    "    print('Optimization complete')\n",
    "    optimizer_runs.append(strategy)\n",
    "    optimization_result = strategy._OptimizerStrategy__result['hedge']\n",
    "    optimization = PositionSet(\n",
    "        date=strategy.initial_position_set.date,\n",
    "        reference_notional=optimization_result['netExposure'],\n",
    "        positions=[\n",
    "            Position(identifier=asset.get('bbid', asset['name']), asset_id=asset['assetId'], weight=asset['weight'])\n",
    "            for asset in optimization_result['constituents']\n",
    "        ],\n",
    "    )\n",
    "    # Applying unadjusted close prices for portfolio analytics to preserve historical price granularity\n",
    "    optimization.price(\n",
    "        use_unadjusted_close_price=True, weighting_strategy=PositionSetWeightingStrategy.Weight, fallbackDate='5d'\n",
    "    )\n",
    "    for p in optimization.positions:\n",
    "        p.add_tag('source', 'Optimization')\n",
    "    # Again using unadjusted prices on source positions for consistency in analytics\n",
    "    latest_source_position_set.price(\n",
    "        use_unadjusted_close_price=True, weighting_strategy=PositionSetWeightingStrategy.Weight, fallbackDate='5d'\n",
    "    )\n",
    "    combined_pset = PositionSet(\n",
    "        date=optimization.date,\n",
    "        positions=[\n",
    "            Position(identifier=p.identifier, asset_id=p.asset_id, quantity=p.quantity, tags=p.tags)\n",
    "            for p in latest_source_position_set.positions + optimization.positions\n",
    "        ],\n",
    "    )\n",
    "    opt_port_manager.update_positions([combined_pset])\n",
    "    if not opt_port_manager.get_portfolio_tree().sub_portfolios:\n",
    "        opt_port_manager.update_portfolio_tree()\n",
    "    start = rebal\n",
    "    rebal = min(max_end, RelativeDate(rebalance_freq, start).apply_rule())\n",
    "    print(f'Scheduling reports to calculate performance till {rebal}...')\n",
    "    opt_port_manager.schedule_reports(start_date=start, end_date=rebal)\n",
    "    time.sleep(2)\n",
    "\n",
    "print(f'Done! Processing completed in {time.time() - start_time} seconds')\n",
    "\n",
    "print(f'As a reminder your optimization can be found in your \"{opt_port_object.name}\" portfolio with id {opt_port_id}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You have successfully completed a basic run of our Quant Backtesting Workflow. \n",
    "\n",
    "For questions, please reach out to [Marquee Sales](mailto:gs-marquee-sales@gs.com)!"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "tutorial-env",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
